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INTRODUCTION 



The Second Annual Presessions of the American Educational Research Association 
included six courses: 



I. Bayesian Statistical Analysis 

Instructor: Donald Meyer 

II. Curriculum Research and Evaluation 

Instructors: Robert L. Baker 

James Popham 
Howard J. Sullivan 

III. Design and Analysis of Comparative Experiments in Education 
Instructors: Gene V. Glass 

Kenneth D. Hopkins 
Jason Mi liman 

IV. Educational Research Management Procedures 

Instructors: Desmond Cook 

Edwin Hindsman 

V. Multivariate Design and Analysis in Educational Research 
Instructors; Joe Ward 

Earl Jennings 
Robert Bottenberg 
Deene Gott 

VI. Research Strategies with Culturally Deprived Children 

Instructor: Martin Deutsch 

The courses were organized and structured to cover the educational research 
terrain as comprehensively as possible and to provide instruction that would be 
suitable for research novices as well as for experienced researchers. Considerable 
effort was given to the selection of the six topics. 

In the fall of 1965 a mail survey was conducted involving a stratified random 
sample of 125 AERA members to obtain suggestions from the membership concerning 
topics. The results of the survey failed to support the hypothesis that persons most 
needing training are most qualified to specify the training they need. However j the 
survey did yield some usable suggestions. Notices in the Educational Researcher also 
solicited ideas. Members of the AERA Executive Committee were very helpful in suggest- 



ing ideas. 



Since official notice of USOE financial support of the project was not received 
until December 1966, publicity for the presessions was limited to announcements in 
the Educational Researcher and to a direct mailing of applications to the AERA member 
ship in mid-December 1966. Altogether 495 completed applications were received. The 
number of applicants exceeded conventional class size for each session. However, 
courses I, II, and IV admitted all applicants indicating the course as their first 
choice. The maximum size of the other courses was established in consultation with 
the instructor of each course. Selections were made by the course instructor. The 
number of persons listing the course as their first choice and the number of actual 

participants are shown below. 

Course Participants Applicants, 



I 


51 


60 


II 


59 


75 


III 


65 


140 


IV 


53 


60 


V 


44 


70 


VI 


50 


90 




322 


495 



Each instructor was totally responsible for the conduct of his course, including the 
evaluations which follow. With one exception, the results of the formal evaluation 
indicate that, in general, the participants both learned from and liked the training 

they received. 

It must be borne in mind, that as a result of extremely late funding and a lack 
of support for program evaluation, adequate staffing, and report preparation, this 
final report varies considerably from presession to presession. Where directors 
found other support for their time and materials, the reports are of excellent quality 
and will hopefully be models for future substantive presession reporting. Each pre- 
session is dealt with as a single chapter and may provide an introduction, a statement 
of objectives, a listing of the presession staff, a listing and/or description of 
participants, a schedule of activities, an example or list of materials utilized, 
presession evaluation and/or evaltiation instruments, and in one case (Presession III) 

a summary and statement of recommendations. 



A brief overall statement of summary and recommendation is made by the 
presessions Program Director, Richard E. Schutz following the final Presession 

report. 



ANNOUNCEMENT 



AMERICAN EDUCATIONAL RESEARCH ASSOCIATION 
RESEARCH TRAINING PRESESSIONS 

February 11-15, 1967 

The American Educational Research Association, aided by a grant from the 
Bureau of Research, U. S, Office of Education, will conduct six intensive training 
courses in various aspeccs of educational research in connection with the 1967 
Annual Meeting. A description of each course is given on the following pages. 
Application : An application form is enclosed. Applications received prior to 

December 1, 1966 will be reviewed and notification of acceptance will be mailed to 
applicants on December 16, 1966. Applications received after December 1 will be 
processed, but acceptance will be contingent upon space limitations and enrollment 
figures prior to December 1. 

Eligibility ; As indicated in the description of each course, AERA membership is not 
an enrollment requisite. 

Location ; Each course will be conducted at a location within New York City with the 
exception of Course III; Design and Analysis of Comparative Experiments in Education, 
which will be conducted at Grossinger's Resort in upstate New York. Rooms for the 
New York City participants will be available at convention v^ates at the Statler 
Hilton Hotel, the site of the Annual AERA Meeting. The total cost to each Course III 
participant for room and meals for the five days at Grossinger’s resort hotel will 
be less than $120. 

Tuition Costs ; None. Purchase of a text may be required for some courses. 

Additional Details ; Lodging applications and further details concerning each 
course will be mailed to participants with notification of aeceptence. Address 
special inquiries to the chairman of the presession committee. Dr. Richard E. Schutz, 
11300 La Cienega Blvd,, Inglewood, California 90304. 



APPLICATION FORM 



AERA 1967 Presession Application 
February 11-15, New York City 



Return completed application to Dr. Richard E. Schutz, Southwest Regional 
Laboratory, 11300 La Gienega Blvd. , Inglewood, California 90304 

Presession Choice 

Indicate your preference using the following code: 

1 - First choice 

2 - Would attend if application accepted 
N - Do not wish to attend 

Bayesian Statistical Analysis 

Curriculum Research and Evaluation 

Design and Analysis of Comparative Experiments in Education 

Educational Research Management Procedures 

Multivariate Design and Analysis in Education Research 

Research Strategies with Culturally Deprived Children 



Nam e 

Last 



General Information 



First Initial 



Age 



Female 



Male 

Home Address 

Number Street City State 

Office Address 

Number Street City State 

Address for Reply (Please check one) : Home Office 

Present Institutional Affiliation (e.g., Harvard, Norfolk Public Schools): 



Employment Responsibilities 
Describe briefly the nature of your present employment:^ 



What percent of your time is allotted to teaching? • 

Wliat percent of your time is allotted to research? . 

Wliich courses do you teach (if any) and at what level (Undergraduate or graduate)? 

Course Level (Circle one) 




Graduate 

Graduate 

Graduate 

Graduate 



Undergraduate 

Undergraduate 

Undergraduate 

Undergraduate 



Approximately how many advisees do you have at the undergraduate level 
the graduate level ? 



at 



Educational History 



ate School _^ear of Degrefe Major 

chool Degree Major 

3 courses in which you have earned either^ undergraduate or graduate credit: 

Undergraduate Graduate 



tatistical Methods 

ducat ional and Psychological Testing .... * . 

esearch Methods . 

sychometric Methods 

alculus 

:athematical Statistics . 

robability Theory . 

Professional and Scholarly Activities 
'our primary research interests? 



-esearch articles which you have authored alone or jointly have been accepted 
.arly (refereed) journal? 

■unded (by USOE, NIMH, Ford Foundation or other granting agencies) research 
aave been completed on which your name appears as either the first or a 

ior? 

presented a paper or appeared. on a panel at the annual convention of either 
i, NSCTE or AACTE within the last three years? NO YES 

:icipate reading a paper at such a convention within the next three 
NO YES 

leir order of importance to you, no more than three professional societies 
lA, American Psychological Association) of which you are a member: 



Additional Information 

offer any information concerning your interests, responsibilities, reasons 
alying, etc. which might assist the committee in reviewing your application. 



ABSTRACTS OF THE 1967 AERA PRESESSIONS 



BAYESIAN STATISTICAL ANALYSIS 
Course I 



Instructors ; Donald Meyer, Syracuse University 

(One additional instructor and one assistant to be added) 



Participants ; 

This session will be open to individuals who hold a doctorate and whose 
academic responsibilities involve designing educational research studies, 
analyzing research data, utilizing research results for making decisions* 
about future educational practice, or teaching courses in educational 
research and statistics. 



Objectives ; Participants will be able to; 

a. Apply a Bayesian approach in designing educational research studies. 

b. Perform Bayesian statistical analyses of educational research data. 

Tentative Structure ; 

Instruction in the form of lectures, prepared handouts, and exercises will 
be announced with the following major topics; 

a. Elementary probability theory and Bayes theorem. 

b. Philosophical foundations of Subjective Probability. 

c. Application of theory to Binomal Sampling -- Estimation and Hypothesis 
Testing. 

d. Bayesian analysis of Linear Models 

e. Miscellaneous Topics 

1) Multiple comparisons 

2) Contingency analysis 





I 



CURRICULUM RESEARCH AND EVALUATION 
Course IX 



!• Instructors ; 

Robert L. Baker, Arizona State University 

James Popham, University of California 5 Los Angeles 

Howard Jo Sullivan, Southwest Regional Laboratory for Educational 

Research and Development 



2 . Participants : 

This session is designed for school and university personnel holding 
doctoral degrees and having responsibility for the formulation and 
evaluation of curricular programs* 

3 • Ob jectives : 

Participants will be able to: 

1. Specify the desired outcomes of curricular programs in terms of 
observable learner behavior* 

2. Describe in operational terms the planned classroom transactions for 
a given curricular program* 

3. Specify appropriate independent and dependent variables for an 
experimental study, and state the specific relationships to be 
investigated in the study, 

4. Select the most valid and practical experimental design for 
investigating the specified relationships, 

5. Construct and/or select valid instruments to measure the extent to 
which the desired outcomes of curricular programs are attained. 

6 . Prepare a research proposal which adequately describes the problem and 
experimental plans. 

4. Structure : 

For each objective listed above, a handout of instructional notes and one or 
more sets of exercises will be prepared. The normal instructional procedure 
will employ in order a series of carefully sequenced instructional sessions 
related to one of the six objectives listed above, exercises designed to 
elicit the behavior specified in that objective, and instructional feedback 
for performance on the exercises. A pretest and post""test will be adminis- 
tered to measure degree of attainment of each of the stated objectives. 



o 

ERIC 







DESIGN AND ANALYSIS OF COMPARATI’S/E EXPERIMENTS IN EDUCATION 

Course III 



1 . Instructors: 



Director: Gene V Glass 3 University of Illinois 

Kenneth Do Hopkins 3 University of Colorado 
Jason Mi liman 3 Cornell Uiiivarsity 



2 - Particip ants : 



This presession will be open to persons x^^ho hold a doctorate in 
education and allied fields and whose primary academic responsibilities or 
interests are in the execution of educational researclu The course it not 
intended for statistics and design specialists but for educational researchers 
x-jlio wish to extend their knoxfledge of design and analysis beyond the point 
it was left after perhaps no more than two statistics courses „ 

Tentative Structure ; 

a„ .Principles of Experimental Design and Analysis (Basic terms 3 control, 
randomization, precision, confounding, interaction, internal and 
external validity, etCc.) 

b« The Mathematical-”S tatis tical Basis of Experimental Design and Analysis 
(Linear model, least-squares estimation, probability distribution, 
hypothesis testing, estimatioia, etc.) 

c„ Rules of Thumb for Writing the ANOVA Table,. (Rules and mnemonic 
devices for determining sources of variation, d„f,, SS, MS, and 
E(MS), in any ANOVA design) 

d. Analysis of Unbalanced Designs (proportional sub-class frequencies, 
missing data, procedures for dealing with disproportionate subclass 
frequencies . ) 

e„ Planned and Post-hoc Comparison (planned orthogonal comparisons, 
multiple comparison techniques, Tuken -and Scheff^ method, etc.) 

f. Consequences of Failure to Meet the Assumption of the ANOVA 

(nominal and actual power and significance levels: non "normality: 
heterogeneous variances: nonindependence). 



educational research management procedures 

Course IV 



1. instructor-. 

T o i-inr-fl f-nT-v .N Austin 9 iexas 



2 , Participants .; 

This session is planned for personnel having anUcfpated 

planning and conducting xi be principal investigators 

that the majority of the P^^icipants will be p^^^^ pLsonnel from 
on individual research 

educational research bureaus and labor atone.. 

3. Obiectives ! Participants will be able to: 

a. Apply program budgeting techniques and cost-benefit analyses in 
planning research projects* 

b. Apply network pJami5.ng techniques in managing research projects. 

c Apply recently developed personnel management procedures in 
administering research projects, 

4, Tpnfative Struc. ^e; 

a Program budgeting concepts and exercises 
b’ Cost benefit analysis principles 

c. Network planning “^/.rel-iallocation of resources 

d* Scheduling concepts and principles ai.tu 

e. Management reports research and development 

f. PERT applications xn edticaf.ionaJ. lesearca 



c. 

;( 



u 



=1 
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multivariate design and analysis in educational research 

Course V ^ 



1. Instructors; 

Joe Ward, Personnel Research Center, Lackland Air Base 
Earl Jennings, University of Texas 

Robert Bottenberg, Personnel Research Center, Lackland Air Base 
Deene Gott, Personnel Research Center, Lackland Air Base 

2. Participants ; 

The session is designed for education researchers who have the basic 
statistical tools in their repertoire, but because of the rapid improvement 
of computer techniques for the systematic organization and analysis of data are 
presently unable to formulate research problems for computer analyses that will 
yield answers to the questions at issue. 

3. Objectives ; 

The primary objective of this session is to assist the participants in 
developing techniques of formulating research problems for computer analyses 
so as to make full use of the multiple linear regression approach. Specifically, 
the participants will be able to; 

a. Define vectors that express the conceptualization of a problem. 

b. Formulate models appropriate for specific problems without conforming 
to experimental designs for which prescribed computational procedures are 

available. 

c. Identify vectors that represent information that has been measured on a 
continuum. 

d. Define vectors so as to express nonlinear and interaction relationships. 

e. Use categorical and continuous vectors in models developed to remove the 
"contamination” of other factors (logic of covariance analysis) 

f. Apply an ambiguous set of rules to the determination of the appropriate 
degrees of freedom to be used with the linear regression model. 

g. Cite novel examples of research problems to which linear regression 
is applicable. 



4, Tentative Structure ; 

This session is designed to develop the appreciation of multiple linear 
regression as a general approach to the formulation and analysis of research 
problems. As such, the activities will be divided about evenly between 
lecture-discussion, laboratory exercises related to the objectives listed and 
exercises related to appropriate computer operations. 

Participants will have direct experience with data processing and computer 
equipment. Each participant will prepare a problem statement which reflects 
acquisition of concepts and development of the attendant techniques that are 
useful in conceptualizing research problems. 
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STRATEGIES OF RESEARCH WITH CULTURALLY DEPRIVED CHILDREN 

Course VI 

! ■ 

5 1 , Instructors; 

Martin Deutsch, New York University 
(One instructor and one assistant to be added) 

2. Participants ; 

This presession is designed for' school and university personnel responsible 
for formulating, planning, conducting, and/or evaluating educational 
programs for culturally deprived children. 

3. Objectives ; 

Participants will be able to; 

a. Describe in operational terms the major causes of cultural deprivation 
and the social conditions associated with deprivation. 

b. Specify the social conditions and causes of deprivation unique to 
each of the major groups of deprived peoples within the society. 

c» Specify practical experimental treatments having a high probability 
q £ reducing or removing causes and conditions of cultural deprivation. 

d. Specify research techniques and strategies appropriate for use in 
experimental research with culturally deprived children, and describe 
the application of the techniques and strategies to existing problems 
of culturally deprived groups. 

e. Prepare a research outline that describes in operational terms the 
existing problem, the dependent and independent variables, and the 
experimental procedures in a proposed experimental study with culturally 
deprived children. 

4. Tentative Structure ; 

The activities for this session will be divided into two types. One type will 
include classroom demonstrations, lectures, and exercises related to the 
objectives listed above. The second set will involve visitations to deprived 
areas of New York City conducted in cooperation with local agencies working 
with deprived groups, and consultation and interviews with representatives of 
such groups. The latter group of activities will provide the participants 
with an opportunity to observe directly the environmental conditions of 
deprivation among several deprived groups, as well as providing the opportunity 
to explore with representatives of deprived groups the underlying causes of 
these conditions and effective techniques of alleviating them. 



PRESESSION I 



BAYESIAN STATISTICAL ANALYSIS 



Director 

Dr. Donald L. Meyer 
Syracuse University 
Syracuse, New York 



I’* 

V- • • • - 

h. 

tr INTRODUCTION 

I, 

J- Bayesian Statistical Analysis was a five-day training. course designed for school 

and university personnel holding doctoral degrees who were engaged in the conduct of 
educational research. The purpose of the Presession was to provide formal training 
in the utilization of Bayes Theorem so that each participant might gain a basic 

" understanding of the philosophical foundations of probability theory, the relation 
of Bayes theorem to elementary probability theory, its application to binomial sampl- 
ing, and its utilization in linear analysis, multiple comparison, and contingency 
analys is . 

; This Presession was one of six intensive training courses in educational research 

sponsored by the American Educational Research Association aided by a grant from the 
Bureau of Research, U.S. Office of Education. The Presession was held on February 11-15 
1967, in connection with the 1967 AEjRA Annual Meeting. 

OBJECTIVES 

The objectives of the Presession were designed to enable participants who satis- 

! 

. factorily completed the program to: 

1. Apply a Bayesian approach in the design of educational research 
studies. 

2. Perform Bayesian statistical analyses of educational research 
data. 

STAFF 

Dr. Donald L. Meyer (Director) 

Associate Professor of Education and 
Psychology 
Syracuse University 

Dr. James Powers (Assistant) 

Syracuse University 

Dr. James Diamond (^sistant) 

Syracuse University 




PARTICIPANTS 



Bayesian Statistical Analysis 



Lawrence Alexander 
535 East 80th Street 
New York, New York 10021 

J. Wm. Asher 

Building *G” S Campus Courts 
Purdue University 
Lafayette, Indiana 47907 

Henry Baich 

5000 N. Willamette Boulevard 
Portland, Oregon 97203 

W. L. Bashaw 
122B Baldwin Hall 
University of Georgia 
Athens, Georgia 

Thomas Blaskovics 
2 Oglebay Hall ' ^ 

Morgantown, West Virginia 

Paul Blommers 

N105 East Hall 

Iowa City, Iowa 52240 

Dale Bolton 

Department of Education 
University of Washington 
Seattle, Washington 

I*awrence Castiglione 
Social Science Building 
Queens College 
Flushing, New York 11367 

Clinton Chase 
Ind ianaUniver s i ty 
Bloomington, Indiana 

Martin Chen 
Dental Health Center 
14th Avenue and Lake Street 
San Francisco, California 

James Conway 
p.O. Box 8065 

Coral Gables, Florida 33124 
James Cooper 

University of New Mexico 
College of Education 
Albuquerque, New Mexico 87106 



William Crawford 
Room 10 Education 
San Francisco State College 
San Francisco, California 94132 

Joseph Fearing 

Bureau of Educational Research 
University of Houston 
3801 Cullen Boulevard 
Houston, Texas 

John Finger 

Rhode Island College 

Providence, Rhode Island 

Robert Fitzpatrick 

135 N. Bellefield Avenue 

Pittsburgh, Pennsylvania 15213 

Vernon Gerlach 
Education 109-C 
Arizona State University 
Tempe, Arizona 

John Ginther 
5835 Kimbark 
Chicago, Illinois 

Donald Green 

Division of Teacher Education 
Emory University 
Atlanta, Georgia 30322 

Bert Grene 

Eastern Michigan University 
Ypsilanti, Michigan 

Naim Gupta 

Ball State University 
Muncie, Indiana 47304 

John Hain 

sue 

New Paltz, New York 

Vernon Hendrix 
211 Burton Hall 
University of Minnesota 
Minneapolis, Minnesota 55455 

Edwin B. Hutchins 
2530 Ridege Avenue 
Evanston, Illinois 



Wilton Jacobseii 
College of Education 
University of Virginia 
Charlottesville, Virginia 

Lee Joiner 
Alumni House 
Hofstra University 
Hempstead, New York 

B. L. Kintz 

West Washington State College 
Bellingham, Washington 

Howard Lamb 
University of Delaware 
Newark, Delaware 

Joy Lindbeck 

Southern Methodist University 
Dallas, Texas 

Robert McMorris 
School of Education 
SUNY at Albany 
Albany, New York 12203 

Dean Meinke 

Indiana State University 
Terre Haute, Indiana 

John Menne 
Counseling Service 
Iowa State University 
Ames, Iowa 

Egon Meimelstein 
Nursery School 
Hofstra University 
Hempstead, New York 

Homer Merrifield 
Science Building 
Ithaca College 
Ithaca, New York 

Estoy Reddin 
School of Education 
Lehigh University 
Bethlehem, Pennsylvania 

J. C. Reid 
Purdue University 
Lafayette, Indiana 

Charles Reimer 

State University College 

Oneonta, New York 13820 



William Rodgers 
5685 Shattuck Road 
Saginaw, Michigan 

Richard Rosomer 
Clarion State College 
Clarion, Pennsylvania 16214 

Joseph Scandura 
Graduate School of Education 
University of Pennsylvania 
Philadelphia, Pennsylvania 19104 

Rodney Skager 

Department of Education UCLA 

Los Angeles, California 90024 

Malcolm Slatker 

SUNY at Buffalo 
Foster Annex 
Buffalo, New York 14214 

Carl Spies 

Bureau of Educational Research- 
Kent State University 
Kent , Ohio 

Charles Stagman 

5200 Rockhill Road 

Kansas City, Missouri 64110 

Marvin Taylor 
Department of Education 
Queens College 
Flushing, New York 

Glen Thompson 
9300 N. Kenton 
Skokie, Illinois 

Bill Turney 

School of Education 

North Texas State University 

Denton, Texas 

James Wardrop 
1404 Regent Street 
Madison, Wisconsin 53706 

Charles Werts 

National Merit Scholarship Corporation 
990 Grove Street 
Evanston, Illinois 60201 



Wilbur Williams 

Eastern Michigan University 

Ypsilanti, Michigan 48197 

George Uhlig 
Garland 108 

University of Wisconsin 
Milwaukee, Wisconsin 53201 
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SCHEDULE OF ACTIVITIES 



Content 

Pre-test. Philosophical discussion of Bayesian and 
classical positions 

Elementary probability theory including definitions 
of important concepts such as independence, sample 
spaces, conditional probability, etc. 

Bayes' theorem, definition and use in simple probabil- 
ity problems 

Laboratory session on problems in the problem set 

Definitions and use of loss and utility functions 

Application of Bayes' theorem and loss functions 
in binomial situations 

7. Natural Conjugate Bayes Densities, definition and 
examples in common sampling situations 

8. Application of Bayes' theorem, loss functions and 
Natural Conjugate Bayes' Densities, to simple situa- 
tions involving normal sampling 

9. Laboratory session on advanced problems in the problem 
set 

10. Advanced normal sampling situations (multiparameter, 

multivariate). Post-test. 



3.1 



Session 

1 . 

2 . 



12 



4. 

5. 

6 . 
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MATERIALS; AN EXAMPLE PROBLEM SHEET AND ANSWERS 



1. You are a counsellor advising a student about his chances of succeeding 
in a certain program. 

a. How might you estimate this chance? 

b. What is the relation between your subjective judgement and 
empirical frequencies? 

c. How is this like or unlike predicting from a regression 
equation? 

2. Suppose you are doing a survey of attitudes and you question people one 
by one. The first ten people answer "yes" to question #1. What is your 
prediction about the next person's response to this question? 

"Evens" comes up ten consecutive times on a roulette wheel. What is 
your prediction about the next spin? 

Comment on similarities or differences between these two examples. 

3. Two urns contain balls as follows; 

Urn 1. 5 green, 3 red 

Urn 2. 2 green, 3 red 

An urn is selected at random and one ball drawn from that urn. 

a. Define a sample space and assign probabilities. 

b. What is the probability of getting a green ball from Urn 1? 
From Urn 2? 

c. What is the probability of getting a green ball? 

4. For the problem above, a ball is selected at random from each urn. 

a. Define a sample space and assign probabilities. 

b. What is the probability of a green ball from Urn 1 and a 
red ball from Urn 2. 

c. What is the probability of getting at least one green ball? 

5. Two six-sided dice, one green and one red, are rolled; 

a. Find the conditional probability of obtaining a sum greater 
than 10, given that the red die resulted in a five. 

b. Find the conditional probability of obtaining a sum less 
than 6, given that the red die resulted in a 2. 

c. Find the conditional probability of obtaining a sum of 7 
given that the red die resulted in a number less than 4. 

6. In number 3 above find the conditional probability of obtaining a red ball 
given that Urn 1 was selected. 

7. In number 3 above what is the probability that the ball came from Urn 1 
given that a green ball was drawn? 
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8. Mr. Smith, having lived in his city many years, estimates the a priori 

probability that today's weather will be inclement is .2. (He thinks today 
will be fair with probability .8). Mr. Smith listens to an early morning 
weather forecast to get some information on the day's weather. The fore- 
caster makes one of three predictions: fair, inclement, uncertain. Mr. 
Smith has made estimates of the conditional probabilities of the different 
predictions given the day's weather as shown below: 



Day's Weather 


Fair 


Forecast 

Inclement 


Uncertain 


Fair 


.7 


.2 


.1 


Inclement 


.3 


.6 


.1 



Suppose the forecaster predicts fair weather. What is the posterior 
probability of fair weather? 

9. Suppose that the reliability of a chext X-ray test for the detection of 
T. B. is specified as follows: of people with T.B., 90% of the X-ray 

examinations detect the disease, but 10% go undetected. Of people free 
of tuberculosis, 99% of the X-rays are judged free of the disease, but 
1% are diagnosed as showing T.B. From a large population of which 
only 0.1% have T.B. , one person is selected at random, given a chest 
X-ray, and the radiologist reports the presences of T.B. What is the 
probability that the person has T.B.? 

10. Suppose we are interested in testing between 

Hq .25 

Hi -75 

and these are the only possible values for Tt, the population proportion 
of "successes". 

a. If a sample of 10 observations (x = success or failure) is 
going to be observed, what would the "classical" decision rule 
be if .05? 

b. Suppose the prior distribution of belief were specified as 
follows: 

Pki 

.25 .50 

,75 .50 

The sample is taken and x = 6 successes are observed. What 
is the posterior distribution for -it? Contrast this with (a). 

11. For Hq: 'TT = *25 and Hi: ft = .75 as in #10, what is the gain in plausibility 

in "decibels" for Hi for each success observed. What is the loss for each 
failure observed? (Note: log 3 - .47712 and log 2 = .30103) 

a. Verify by using net gain in decibels that the Bayes factor 
for Hi is slightly less than 10 if x = 6 and N = 10. 

b. How many successes would have to be observed in 14 trials to 
make the Bayes factor at least 100 in favor of Hi? 
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T'and’^.lO resprct;tely;'°(ms-co;id"^: p^pC^a^iorro Zr^'ior 

■L. ’ u u•1^f-^7 of heads on a toss of an unknown coin.) ^ . 

the probability of heaas on a u . ^ number of heads in 

a. Find the predictive probabilities tor x 
10 tosses. 

b. Would you be surprised if x =9- jq against H, ;-n ^ -50 

o. What is the likelihood ratio for Hq . T' “8 1 

if X ® 9 successes are observed? 

7*^ * N ^ 10 1^ ^ ^0ClSlOtl 

Tf1Me?t'H^f“x;'l“wta^L^thea^nd« «rors? If ’prior probabilities 
and^r^ aL° equal, what is the ia.plied loss ratio! 

14. For Hq : = -^0. «i: « = -80 the following loss table applies. 

Truth; 



U 



13. 



15. 



15. 



17. 
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H, 



Dec is ion 



Ho 



^0 



H, 



Hi 8 



a. Find the regret table t-rlals? 

c.‘ S^P(Ho)^=^-3''aJS^P(Hl^ *7 ^hat is the Baye's rule for n = 2? 

For n = 10? 

Find the classical confidence interval for s.if W = 10 and x = 3 using 
1 - a = .90. 

,0. ..V. . ~ 



• Hi cate the 507o of patients having spinal anesthesia complain 
Hospital records indicate the OU/o p • „i fluid pressure has been 

of post-operative headaches. ^ ^^^"^"PHee^s to "indicate that this may 

observed in some “ causing headaches. From talking with patxents, 

not be an important factor in researcher feels quite strongly 

doctors, etc. and reading resear uatient. The researcher designs 

that a major factor is the attitude of the l_^^ 

a study involving 10 patients w minimize the headache which 

ol°c'url‘'a7rsrinaran:rthesia. The -ear^fe^^^^ the^Xh^ 



J^_ 




.50 


. 10 


.60 


.20 


.70 


.40 


.80 


.20 


.90 


.10 



This is assuming these are the only 

report headaches. What is the posterior distribution for rf • 






17. (cont.) 



'a. Graph the prior and posterior distributions. 

•b. What trauld the "classical" decision rule be for testing 
Hq : ^ = .50 agains t at^=.05? 

c. What is the posterior probability of Hq? 

d. What was the predictive probability of * = 8? 

e. Suppose a further sample of 10 is contemplated. What is the 
predictive probability of observing another x = 8? 



18. Suppose you estimate that the mean of your prior distribution for n' is .20 
and probability of tT ^ .40 is .05. 



a. Use our charts to find the parameters of a Beta density to 

represent the prior. 

b. If a sample of 20 is taken and x = 3 observed, what is the posterior 
distribution? 

c. Using our charts, construct a .95 posterior interval for it . 

d. What is your point estimate of fT ? 

19. Suppose your are "indifferent" about it . What is your posterior beta density 
• for Tt if N = 20 and x = 3? What is your best point estimate? 

20. If your prior mean of rf is ,50 and your prior probability of .40^ if < .60 
is .50, what prior beta distribution would apply? 

21. For the prior in #20, suppose 

.50, 

Hi:ir> *50 

with the following loss table: 



"truth" 

Hn Ho 

Decision 0 ^ 

Hi 2 0 

a. If you sample N = 10, what is the Baye*s decision rule? 

b. What is your expected risk: 

22. For the prior in #20, what is the predictive probability of x = 0, 1, 2 
in a sample of N = 2? 

23. For #20, what is the expected number of successes for a sample of 10? 

24. Suppose you feel that the probability of Tt being exactly .50 is .90. 
The remaining probability is spread over the range of 'IT' like a beta 
distribution with n* = 30 and r* = 15. 

a. What is the Baye's factor for Ho;ir= .50 against Hi;7T# .50 
if N = 10 and x = 10 observed? 
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25 Suppose you sample from a binomial population until number of 
successes divided by t^e number of trials 

N = 1,000,000 and x/n = .50, you stop. Is this stopping rul 
"informative*'? 

26 An investigator wishes to ascertain if the sixth grade children of low 

Zb, in a laree city are below normal in intelligence. From 

rspe^merunivLse ff'chiidren of a low Income S-cP.^he investigator 
takL a random sample of 100 children and admini^ers the Stanford-Bi . 

The sample mean is found to be 96. Assume that <r- 256. 

a. Olassical Test the hypothesis that Hq: A'= 100 against Hi: 

A <100. Construct a .95 confidence interval tor 

b. Bavesian— -The investigator has some suspicion thatp^is below 

100 His prior for is approximated by a normal distribu i 
Tt; mea^ =92 and standard deviation of 8. Find the^posterior 

distribution of/^. What is the P(Hq) where i ■ 

Conftiuct a .95 probability interval about thS posterior mean. 

27. For #26, if a further sample of 25 were taken and x - 100, what would 
the final posterior for be? 

1 i<- Jkoi csiirnrisine'^ What was the predictive probability 

28. Was the sample result in #27 surprising, wuai. wa r 

of X ^ 100? 

99 A prior for^ = “ /^2 has a mean of zero and is such that is 

rougUy 2/3. If 100, what normal distribution would approximate the 

prior. 

90 For #29 if we do normal sampling with H = 10 in each treatment group and 

30. ^ is the posterior distribution of ^ 1 

31. What is the posterior probability that'^= 0? (For #30) 

32. For a normal distribution, you estimate that P(v>6) = ^25 and P(cr> 4) = .50. 
What prior gamma - 2 distribution for l/„2 would apply? 

33 continuing With #32, you estimate that>/= 100 with P(96</^<104) = .50 if 
^ 16. Using the normal distribution to represent your conditional 

distriLtions liven </(or what is the marginal prior for/2. 

34. For #32 and 33, if x = 1^2, S* = 20 and N =. 10, describe the posterior 
distributions for// and o' # 
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3. 



4. 



5. 

6 . 

7. 

8 . 



Answers 

Problem Sheet on Bayesian Statistics 



(b) 5/16, 1/5 

(c) 41/80 

(b) 15/40 

(c) 31/40 
1 / 6 , 1 / 2 , 1/6 
3/8 

25/41 

56/62 



9. 90/1089 



10 . 



12 . 



13. 

14. 




(a) reject if x - 6 

(b) P(tt = .25 I X = 6) = .099 
P(^ = .75 I X = 6) = .901 




P(x) 
.001 
.011 
.045 
.118 
.204 
.244 
.204 
. 118 
.045 
.011 
.001 



(b) Yes 



8.6 to 1 



truth 



Ho 



Hi 




Minimax rule: reject if x ~ 2 

Bayes rule: reject if r = 0 



(c) 




.400 



i.e., do not sample 



15. C(.03<rr-^ .74) “ .90 

16. Approximately: 

P(.13 < TV < .58) = .90 

mean a .33 
M.L.E. =* .30 



Assuming 


X = 7 observed 


TT 


11 


.50 


.056 


.60 


.207 


.70 


.515 


.80 


.195 


.90 


.027 



26. (a) 2 = -2.5 

C (92. 86-«i/-'< 99.14) 

(b) mean of posterior = 94.96 
variance - 1.94 



P(H£,) = P(z = 3.63) 
P(92.24< P < 97.68) = .95 



27. normal (95.48, 1/.615) 

29. m' - 0, V* = 25 

30. m' - 3.89 
v' = 11.11 

31. P(^- 0) = P(2 « 1.17) = .121 
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EVALUATION AND DIRECTOR'S COMMENTS 

Fifty-one participants attended the presession. Most areas of the country were 
represented. States represented included California, Texas, Indiana, Minnesota, 

Virginia, Delaware, New York and New Mexico among others. Both large and small 
universities were also represented. 

Classes were held daily from 9:00 A.M. to 4:30 P.M. with a coffee break In the 
morning and afternoon and an hour and a half break for lunch. 

• The facilities were not adequate as regards; blackboards, lighting and work space. 

The blackboards were old and difficult to see and write on. Tables should be provided 
so that students have a hard surface to write on in addition to work space. 

A twenty-item pre-test on "classical" statistics was given on the first day and 
a nineteen-item post-test covering Bayesian principles was given on the last day. 

Scores on the pre-test ranged from 1 to 17 with a mean of 9 and a standard deviation of 
3.6. This indicates the wide heterogeneity of background represented. Consequently, 
we decided to teach somewhat slower than originally planned. The post-test scores 
ranged from 3 to 15 with a mean of 8 and a standard deviation of 3.2. Nothing 
should be inferred from the fact that the post-test mean was lower than the pre-test 
mean since the the two tests were, for the most part, different. were somewhat 
disappointed with the results of the post-test, but many of the items were difficult 
inasmuch as they covered concepts which would be interpreted differently in Bayesian 
theory as contrasted with classical theory. An item analysis of the post-test revealed 
that 68% of the students answered correctly on the item regarding the relation between 
classical and Bayesian theory. On three items regarding uses of Bayes theorem involving 
direct calculation and application to posterior distributions, the percentages answer- 
ing correctly were 42%, 52% and 52%. This was somewhat heartening as we could assume that i 
prior knowledge on this material was near zero. ; 

I 
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An anonymous attitude survey was also administered on the fxnal day 
responses weret 

Do you anticipate doing more study of Bayesian statistics? 




definitely yes 
probably 
probably not 
definitely no 



25 

9 



17 

11 



If you teach statistics or research methods, will you introduce some 
Bayesian concepts to your students. 

definitely yes 
probably 

I do not teach statistics 7 

1 - r>f T^avesian statistics to your current 

DO you think that the addition research work? 

statistical knowledge will aid you in your i 



definitely yes 
probably 
probably not 



21 

12 

1 



, -11 Ravesian statistics or concepts in your 

Do you think you will apply Bayesian 

research work? 



definitely yes 
probably 



13 

19 



DO you think you have a better understanding of -classical" statistics 
as a result of the presession? 



defintely yes 
probably 
probably not 
definitely no 



26 

6 

2 

1 



„ »• “■ 
uu. .b». ». «.»"»•• •• “>• ““ ^ 

^ of the papers included responses such as "well-organised", "excellent 
Ninety-five percent of the papers in»-j.u ^ 

t- vtaA no pi*iticisin« but the other ei§ ty 
teaching", "stimulating". About twenty percent had 

u Hcbould have had more material on dittos", "more dis- 
percent made comments such as, shoul 

• and other comments relating to physical conditions 

cussion", "too much to assimilate , and ocner ^ 



as mentioned previously. 
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Director 

Dr. Robert L. Baker 
Arizona State University 
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INTRODUCTION 



Curriculum Research and Evaluation was a five-day training course designed 
for school and university personnel holding doctoral degrees and having responsib 
ility for the formulation and evaluation of curriculum programs. The purpose of 
the Presession was to provide formal training in the necessary methodological 
skills to formulate, initiate, conduct, evaluate and report experiments on school 
learning and instruction. 

This Presession was one of six intensive training courses in educational 
research sponsored by the American Educational Research Association aided by a 
grant from the Bureau of Research, U. S. Office of Education. The Presession was 
held on February 11-15, 1967, in connection with the 1967 AERA Annual Meeting. 

OBJECTIVES 

The objectives of the Presession were designed to enable participants satis- 
factorily completing the program to: 

1. Specify the desired outcomes of curricular programs in 
terms of observable learner behavior. 

2. Describe in operational terms the planned classroom 
transactions for a given curricular program. 

3. Specify appropriate independent and dependent variables 
for an experimental study, and state the specific relation- 
ships to be investigated in the study. 

4. Select the most valid and practical experimental design for 
investigating the specified relationships. 
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STAFF 



Dr. Robert L. Baker (Director) 

Arizona State University and the 
Southwest Regional Laboratory for 
Educational Research and Development 

Dr. W. James Popham (Instructor) 
University of California at Los Angeles 

Dr. Howard J. Sullivan (Instructor) 
Southwest Regional Laboratory for 
Educational Research and Development 

Miss Carolyn Wilkerson .(Assistant) 
Southwest Regional Laboratory for 
Educational Research and Development 



Miss Carole Waina 

Southwest Regional Laboratory for 
Educational Research and Development 



PARTICIPANTS 
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Curriculum Research 



and Evaluation 
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Adear, Mary H. 

Pennsylvania State University 
University |*ark, Pennsylvania 

Alston, Gilbert R» 

University of New Hampshire 
Durham, New Hampshire 



Bailey, Benjamin H. 
College of Education 
West Virginia University 
Morgantown, West Virginia 



Barknard, James Gerald 
New York University 

Benjamin, Richard Co 
University of Michigan 
Ann Arbor, Michigan 

Brodt. Marjorie B. 
Mecklinger Public Schools 
Charlotte, North Carolina 

Brown, Marjory R. 

City College of New York 
New York 



Burke, James M. 
Connecticut Street 
Education Department 
Connecticut 

Clegg, Ambrose A., Jr. 
University of Washington 
Seattle, Washington 




Dale, Ruth E. 
Queens Co 11 lege 
Flushing, New York 







Bidwell, Wilma 
SUNY at Albany 
Albany, New York 



Dopyera , John F • 
Syracuse University 
Syracuse, New York 



Durkin, Dolores 
University of Illinois 
Urbana, Illinois 



Eagle, Normal 

Mount Vernon Public Schools 
Mount Vernon, New York 



Eash, Maurice 
Hunters College 
New York, New York 



Ellis, Joseph R. 

Northern Illinois University 
De Kalb, Illinois 



Finley, Carmen J. 

Somes tona County Public Schools 
Santa Rosa, California 



Fisher, Ijourie S. 

Miami Dade Junior College 
Miami, Florida 



Foster, Judilynn T. 
Riverside Public Schools 
Riverside, California 



Grobman, Hulda 
New York University 
New York, New York 



Hauenstein, Dean A. 
Chicago State College 



Heisey, Daniel J« 

University of New Hampshire 
Durham, New Hampshire 



Henjum, Raymond H. 
Montana State University 
Bozeman, Montana 



Hines, Vince Ao 
University of Florida 
Gainesville, Florida 



Hinley, Reginald To 

North Texas State University 

Denton, Texas 76203 



Howell, Edgar M. 

Temple University 
Philadelphia, Pennsylvania 



Hull, William L. 

Oklahoma State University 
Stillwater, Oklahoma 



Erwin, Clara 

Wayne State University 

Detroit, Michigan 



Johnson, John Lo 
Syracuse University 
Syracuse, New York 



Kreutz, Shirley M. 
University of Nebraska 
Lincoln, Nebraska 



Lamb, John C. 

Harrisburg Community College 
Harrisburg, Pennsylvania 



Leibert, Robert E. 
University of Missouri 
Columbia, Missouri 



Spigle, Irving So 
Parkforest Public Schools 
Chicago 

Sterner, ’William F. 

Rutgers University 

New Brunswick, New Jersey 



Lowe, William T. 

Cornell University 
Ithaca, New York 

Maher, Allan E. 

Unqua School 
Massapeq^a, New York 

Merz, William R. 

Clark County School District 
Las Vegas, Nevada 

Morris, Earl W. 

Central Midwestern Educational Laboratory 
St. Louis, Missouri 

Ober, Richard L. 

University of Florida 
Gainesville, Florida 

0* Connell, Ernestine 
Old Dominion College 
Norfolk, Virginia 

Owens, Robert G. 

Brooklyn College 
Brooklyn, New York 

Pfeiffer, Isobel 
University of Akron 
Akron , Ohio 

Remick, Edward L. 

Lansing School District 
Lansing, Michigan 

Romberg, Thomas A. 

University of Wisconsin 
Madison, Wisconsin 

Rubinowitz, Arthurea B. 

Fairfax County School 
Fairfax, Virginia 

Sherron, Gilbert 

Centrad Midwestern Educational Laboratory 
St. Louis, Missouri 

Zwirt, Marsha D. 

Middle Tennessee State University 
Murfreesboro , Tennessee 

Wairke, Oman F. 

Ohio State University 



Streich, William H. 

Farmington Public Schools 
Farmings ton, Connecticut 

Sullivan, John V. 

Kansas State Teachers College 
Importa, Kansas 

Taylor , Aus tern 

Virginia State ’Education Department 
Richmond, Virginia 

Thomas , Elizabeth 
University of Iowa 
Iowa City, Iowa 

Thomas , Don 

Central Midwestern Educational Laboratory 
Stc Louis, Missouri 

Veal, L. Ramon 
University of Georgia 
Athens , Georgia 

Wallace Gayle R. 

Oklahoma State University 
Stillwater, Oklahoma 

Wasik, John 

Montgomery County Public Schools 
Rockville, Maryland 

Wilson, Brent G. 

University of Iowa 
Iowa City, Iowa 

Wing, Lucy F. 

University of Maryland 
Hyattsville, Maryland 

Wonder ly, Donald L 
Kent State University 
Ohio 

Zink, Theodore M. 

Glassboro State College 
Glassboro, New Jersey 
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DESCRIPTIVE ANALYSIS OF PARTICIPANTS 



Number of participants from each state 

State and Institution 
Represented 

California 

Riverside Public Sch. 

Sonoma Public Sch. 

Connecticut 

St. Educ. Dept. 

Florida 

Miami-Dade J.C. 

U. of Florida - 2 
Georgia 

U. of Georgia 
Illinois 

CMREL - 2 
N. Illinois U. 

Park Forrest Public Sch. 

U. of Illinois 

Iowa 

U. of Iowa - 2 
BCansas 

Kansas St. Teachers Col. 
Maryland 

Montgomery Co. Public Sch. 

U. of Maryland 
Massachusetts 

Ho 11 is ton Public Sch. 

U. of Massachusetts 
Michigan 

Lansing School Dist. 

U. of Michigan 
Wayne State U. 

Missouri 

U. of Missouri 
Montana 

Montana State U. 

Nebraska 

U. of Nebraska 
Nevada 

Clark Co. School Dist. 

North Carolina 

Mecklinberg Public Sch. 

New Hampshire 

U. of New Hampshire - 2 
New Jersey 

Glassboro State Col« 

Rutgers 

) Trenton St. Col. 



Number (N=5^ 
2 

1 

3 

1 

5 



2 

1 

2 

2 

3 



1 

1 

1 

1 

1 

2 

3 



o 

ERIC 



Percent 

3 

2 

5 

2 

8 

3 

2 

3 

3 

5 

2 

2 

2 

2 

2 

3 

5 



r 



Number 



Percent 



State and Institution 
Represented 

New York 

Brooklyn College 
Cornell U. 

CUNY - 2 

Massapegua Public Sch, 

Mt. Vernon Public Sch. 

New York U. 

Queens College 
Syracuse U. - 2 

Ohio 

Kent State U. 

Ohio U. 

Ohio State U. 

U. of Akron 
Oklahoma 

Oklahoma State U. - 2 
Pennsylvania 

Harrisburg Community Col. 
Pennsylvania State U. 
Temple U. 

Tennessee 

Mid-Tennessee State U. 
Texas 

North Texas State U. 
Virginia 

Fairfax Co. Public Sch. 
Old Dominion College 
St. Educ. Dept. 

West Virginia 

West Virginia U. 
Wisconsin 

U. of Wisconsin 



11 



4 



2 

3 



1 

1 

3 



1 

1 



19 



7 



3 

5 



2 

2 

5 



2 

2 



Age Groupings 
Age Bracket 



Number (N=>58) Percent 



55 and up 

50-54 

45-49 

40-44 

35-39 

30-34 

25-29 

20-24 



2 


3 


4 


7 


10 


17 


9 


16 


18 


31 


12 


21 


2 


3 


1 


2 
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Number of male and female participants 

Male . Total Number 

43 70% 61 

Female 

18 . 30% 



Year of bachelor degree and major field 



Year 


Number 


Percent 


Major Number 


Percent 




(N=58)“ 










1930-34 


2 


3 


Elem. educ. 


7 


12 


1935-39 


5 


9 


English 


7 


12 


1940-44 


5 


9 


(and speech) 






1945-49 


11 


19 


Mathematics 


7 


12 


1950-54 


16 


28 


, History 


6 


10 


1955-59 


15 


26 


(and pol. sci.) 






1960-64 


4 


7 


Social studies 


6 


10 


1965— 


0 


0 


Psychology 


5 


9 








Chemistry 


3 


5 








Industrial arts 


3 


5 








Biology 


2 


3 








Sec, educ. 


2 


3 








Social psych. 


2 


3 








Special educ. 


2 


3 








Agriculture 


1 


2 








Art 


n 

X 


2 








Curriculum 


1 


2 








Home econ. 


1 


2 








Languages 


1 


2 








Sociology 


1 


2 


Year of doctorate 


and major field 








Year 


Number 


Percent 


Major Number 


Percent 




wStr 










1930-34 


0 


0 


Educ, psych. 


10 


19 


1935-39 


1 


2 


Curriculum 


6 


11 


1940-44 


0 


0 


Educ, admin. 


6 


11 


1945-49 


1 


2 


Sec, educ. 


6 


11 


1950-54 


1 


2 


Elem. educ. 


5 


9 


1955-59 


9 


17 


Mathematics 


4 


8 


1960-64 


14 


26 


Industrial arts 


3 


6 


1965 


11 


21 


Counseling 


2 


4 


1966 


13 


25 


Science 


2 


4 


1967 


3 


6 


Social psych. 


2 


4 








Agriculture 


1 


2 








Art 


1 


2 








Exceptional child 


1 


2 








Home econ. 


1 


2 








Political sci. 


1 


9 








Special educ. 


1 


2 








Technology 


1 


2 



r" 



Courses taken 
Name 



(N=58) 



Statistical Methods 

Educational and Psychological Tasting 
Research Methods 
Psychometric Methods 
Calculus 

Mathematical Statistics 
Probability Theory 



Position 

t 

Type 

Education professor 
Educational psychology professor 
Evaluation and research 
Student teacher instructor 
Testing service 
Counselor education 



Number 

(N=58) 

. 10 
9 
8 
2 
2 
1 



Research and planning head 
Director - training school 
Industrial arts professor 
Mathematics education professor 
Psychology professor 
Agriculture education professor 
Art education professor 
College professor (unspecified) 
Economics education professor 
English education professor 
Political science professor 
Reading education professor 



3 

3 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 



Curriculum supervisor 
Science-math, department head 
Coordinator of program imple- 
mentation 

Elementary school principal 
Special education instructor 



No. 


% 


No. 


% 


18 


31 


54 


93 


24 


41 


41 


71 


10 


17 


51 


88 


2 


3 


23 


40 


18 


31 


3 


5 


12 


21 


8 


14 


8 


14 


13 


22 



Percent 



17 

16 

14 

3 

3 

2 

5 

5 

3 

3 

3 

2 

2 

2 

2 

2 

2 

2 

3 

3 

2 

2 

2 



o 

ERIC 



Percent of time teaching and in research work 



Percent 



Teaching (N=59) 

Number % of Participants Percent 



Research 

Number % of Particin.-'.nt.s 



90-100. • 


4 


7 


90-100 


5 


8 


80-89 V 


2 


3 


80-89 


0 


0 


70-79 • 


6 


10 


70-79 


4 


7 


60-69 


4 


7 


60-69 


4 


7 


50-59 


13 


22 


50-59 


13 


22 


40-49 


1 


2 


40-49 


2 


3 


30-39 


5 


8 


30-39 


6 


10 


20-29 


3 


5 


20-29 


9 


15 


10-19 


0 


0 


10-19 


3 


5 


0- 9 


20 


36 


0- 9 


13 


22 



Number of advisees 


Undergraduate 


(N=59)' 


Graduate 




Number Percent 




Number 


Percent 


125 up 1 2 


125 up 


0 


0 


100-124 1 2 


100-124 


0 


0 


75- 99 1 2 


75- 99 


1 


2 


50- 74 2 3 


50- 74 


2 


3 


25- 49 8 14 


25- 49 


4 


7 


1- 24 4 7 


1- 24 


22 


37 


0 42 71 


0 


30 


51 


Primary research interests 


Field 


Number 

(N=70) 


Percent 




Curriculum 


33 


47 




Measurement and evaluation 


10 


14 




Learning theory 


8 


11 




Programmed instruction and AV 


5 


7 




Teacher education 


4 


6 




Elementary education 


2 


3 




Mental retardation and 2 

emotionally disturbed children 


3 




Teacher-student relations 


2 


3 




Culturally deprived 


1 


1 




Guidance 


1 


1 




Proposal writing 


1 


1 




Reading 


1 


1 





Number of journal articles 



Number 



7 up 
6 
5 
4 
3 
2 
1 
0 



No. of participants 
(N=58) 

6 

0 

2 

3 

6 

8 

7 

26 



Percent 

10 

0 

3 

5 

10 

14 

12 

45 



Number of funded research projects 



Projects 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 



No. of participants Percent 
(N=58) 

0 
0 
0 
0 
0 
0 
1 
1 
7 

11 
38 



2 

2 

12 

19 

65 



Number reading a paper at convention 



Yes 

No 



Present 



Number 

17 

41 



Percent 



29 

71 



Future 

Number Percent 
Yes '33 62 

No 15 28 

Don't know 5 9 



Rank order of professional societies 



First (N«58) Second (N«58) 



Tvne 


Number 


Percent Tvne 


Number 


Percent Type 


AERA 


38 


66 


ASCD 


12 


21 


PDK 


APA 


4 


7 


AERA 


9 


16 


NCME 


APGA 


2 


3 


NOME 


8 


14 


AERA 


ASCD 


2 


3 


APA 


6 


10 


CEC 


(AAMD.AEA.CERA, 




IRA 


3 


5 


APA 


IRA.NM.NAITE 






NSSE 


3 


5 


AAAS 


NASSP.NCSS.NCTE, 




NARST 


2 


3 


DESP 


nctm,nea,pdk) 


12 


21 


NCTM 


2 


3 


NCSS 








PDK 


2 


3 


Mis cl 








(Mis cl.) 


11 


19 





Third (N=53) 

Number T?erc» nt . 



8 

5 

3 

3 

2 

2 

2 

2 

24 



16 

10 

6 

6 

4 

4 



47 
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DESCRIPTION OF PROCEEDINGS 



n 1 - Organization and Pretest 

■he first session began at 8:30 Saturday morning, February 11, with an introduc 
: the staff and an overview of the purposes of the Presession by Dr. Baker. Two 
Its were given to the participants: (1) Schedule and Objectives (Appendix Bj , 

i) program Overview (Appendix C). The participants were then given the Pretest 
idix D) which took the remaining part of Session 1. The results are presented 
2 next major section of this report. 



L) 






3 n 2 “ Types of Research Studi es and Variables 



an 3 - Research Studies and Inference. - Dr. Sullivan, instruct or 

The content of the first instructional sessions were to develop the following 



research concepts! 



1. variable 

2. criterion variable 

3. variate 

4. manipulable variable 

5 . non-manipulab le 
variable 

6. status study 



7. associational study 

8. experimental study 

9. independent variable 

10. dependent variable 

11. statements of association 

12. statements of causality 



The content was presented in lecture form accompanied by a handout of exercises 
OTdix D). At the conclusion of Session 3, the Summary Sheet (Appendix E) of 
iture content was distixbuted. 



Ion 4 - Educational Outcomes - Dr. Popham, instructor 
The content of this session was to develop the necessity for the use of behavioral 

d operational language in the specification of goals. 

A behavioral objective filmstrip (V3MCEI) with taped commentary was viewed by the 

rticipants who responded to questions using an answer sheet (Appendix F). 

Establishing minimal performance standards for individual and class levels 

SO discussed. 



was 
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Sessions 5, 



10, 15, 



and 20 - Informal Discussion 



Participants were allowed to use these sessions for independent study or other 
activities. 

Session 6 - Research Inferences - Dr. Sullivan, instructo r 

The uses of the various types of studies were discussed in this session, as well 
as the types of statements that one can make about data based on the type of study 
used. One of the purposes of the Presession had to be re-emphasized as being concerned 
with what one can say and do to implement one’s objectives rather than aiding in the 
making of value judgments of particular objectives. 

Session 7 - The Classroom and Experimental Error: “ Dr. Baker, instructor 

The content of this session, which was concerned with Objective 4, was in the 



form of three self-contained packages: 

1, Experimental Design in Educational Research (Appendix G) 
which was an advance organizer for packages on experimental 

design. 

2. Experimental Error - Internal and External Validity (Appendix H) 
which dealt with the presence of variability in the results o 
studies. The content outline is as follows: 

I. Internal and External Validity 

II. Threats to Internal Validity 

A. History 

B. Maturation 

C. Testing 

D. Instrumentation 

E. Regression 

F. Selection 

G. Experimental Mortality 

H. Select ion -maturation Interaction 

III. Threats to External Validity 

A. Testing and X- inter act ion 

B. Selection and X-interaction 

C. Reactive Arrangements 

D. Multiple X Interference 

3. Methods for Reducing Error (Appendix I) - which had to do with 
randomization, control groups, and other ways to reduce experi- 
mental error. The content outline is as follows: 

I. Randomization 

II. Control Groups 













III. 



other Ways to Reduce Experimental Error 

A. Refinement of Techniques 

B. Blocking 

C. Adjustments in Statistical Analysis 

D. Number of Experimental Units 

E. Factorial Experimentation 

The last two packages contained exercises with chemically pre-treated answer sheets 



Session 8 - Alternative Designs - Dr. Baker, instructor 

After a short discussion on questions raised in regards to the content of the 
previous session; another self~contained package with exercises and chemically pre 
treated answer sheets was presented to the participants. This package, called 
Alternative Designs (Appendix J) , had the following content outline! 

I. Introduction 

II. The Design *s Critical Features as Illustrated by the 
Classical Experimental Design 

III. Alternative Designs 

A. Repeated Observation Designs 

1. Time series design 

2. Varying treatment time series design 

3. Multiple time series design 

4. Equivalent time samples design 

5. Equivalent materials samples design 

B. Pretest-Posttest Designs 

6. Nonequivalent control group design 

7. Separate sample pretest-posttest design 

8. Separate saniple pretest-posttest control group 

design 

C. (9) Institutional Cycle Design 

IV. Selecting the Best Design 



Session 9 - Laboratory-Discussion 

This session was devoted to small discussion groups and individual study for 
working on packages. The group discussion topics were: 

1. Evaluating Teacher Education Sequences 

2. Validating Instructional Products 

3. Evaluation in Schools using Flexible Models 









Session 11 - Classroom Transactions - Drs. Popham and Sullivan, instructors 

Dr. Popham aUt^pted to describe in operational terms the planned classroom 
transactions through the use of a filmstrip, ''Improved Educational Programs" and 
taped commentary. Participants responded to the audiovisual media by providing 
answers. (Appendix L is the answer sheet.) Four central ideas an educator must 
consider in proposed ins|^mctional improvements were stressed as follows; 






4- 



1. Independent variable 

2. Reproducibility 

3. Justification 

4. Evaluation 

A great amount of time was spent in group discussion on objectives. 

\ 

\ 

Session 12 - Criterion Measures - Dr. Pojpham. instructor 

A three rubric classification scheme for categorizing various types of 
educational criterion measures was described. This classification model was pre- 
sented through transparencies on the overhead projector and a taped lecture. The 



model is as follows: 




The participants responded through the use of an answer sheet (Appendix M). Handouts 
included Criterion Measures Examples (Appendix N) and Metfessel and Michael's "A 
Paradigm Involving Multiple Criterion Measures for the Evaluation of the Effectiveness 
of School Programs" (Appendix 0). 

Session 13 - Alternative Designs - Dr. Baker, instructor 

This session was used as a review of the content of packages on experimental 
design. Dr. Baker cleared up misunderstandings, re-explained some material, and gave 
several examples on the chalkboard. The content mainly consisted of internal and 

external validity and design paradigms. Some questions on statistics were also 

considered, 

o 

ERIC 



Session 14 - Laboratory-Discus s ion 



This session, like Session 9, was devoted to small discussion groups and individi 
study for working on packages. The topics were as follows: 

1. Estimation vs. Hypothesis Testing (Baker) 

2. Values and Weaknesses of Taxonomies (Sullivan) 

3. Unobtrusive Measures (Popham) 



Session 16 - External Control Procedure_s - Dr. Sullivan, instructor 

Dr. Sullivan described operationally appropriate "contingency management" 
procedures. His discussion stressed the following ideas: (1) often in a research 

project too little attention is given to the transactions which actually occur in the 
classroom; and (2) techniques which can be employed to monitor treatment delivery in 

the classroom to insure intended transactions are: 

a. Pre- training of teachers in which they actually 

practice using the materials 

b. Frequent review conferences 

c. Record-keeping 

Handouts were a description of Field Tryout, Reports (i^pendix P) and examples of 
Classroom Observation Record (Appendix Q) and Lesson Record Sheet (i^pendix R). 



Session 17 - Criterion Measures - Dr. Popham, instructor 

Various kinds of evaluative procedures which are suitable for different purposes 
were described. Also, such phrases as formative evaluation, summative evaluation, 
norm referenced measures and criterion referenced measures were discussed. By pre- 
senting an empirical instruction paradigm. Dr. Popham showed how evaluative measures 
were dependent on prior aspects, especially behaviorally stated important objectives. 

EMPIRICAL INSTRUCTION PARADIGM 




Section 18 - Components of the Researc h Proposal - Dr. Baker, instructor 

Proposal formulation and planning were discussed in this session (which was 
shortened). The basic components of a sound research proposal were described, 
(a) differences in labels attached to components in varied proposal formats. 



(b) differences in ways of sequencing and/or organizing component in overall pro- 
prosal structure. Lastly, Dr. Baker explained the conditions that must be met for 
each basic component of a sound proposal. The above points were covered extensively 
in a handout. The Ingredients of the Research Proposal (Appendix S) . 

Section 19 ” Posttest 

It was announced several hours earlier that the posttest would be given during 
this session. Participants were given approximately an hour to study prior to this 
session. The post test may be found in Appendix V. The posttest results are shown 
in an analysis in the next major section of this report. 

Sessesion 21 - Description of the Research Problem an d Experimental Plans 
Dr . Baker , ins true tor 

Dr. Baker discussed in more detail the components of the research proposal 

and answered additional questions concerning setting up one*s experimental plans. 

An article by Schutz and Baker, "The Experimental Analysis of Behavior in Educational 

Research" (Appendix T) was distributed. 

Session 22 - Summary 

Dr. Baker unveiled the posttest results, comparing them with the pretest results 
Participants received feedback on their specific pretest and posttest scores. Non- 
intellective evaluative devices were distributed and completed- -the Participant 
Critique Form (Appendix W) and the Presession Rating Sheet (Appendix X). Results 
of these devices are included in the Evaluation section. 



EVALUATION 



This section includes the data of the participants' scores on the 
pretest and posttest, two non- intellective devices — Participant Critique 
Form and Presession Rating Sheet, and participant responses on some of the 
mastery tests. Lastly, the staff assistants' appraisal and recommendations 
are presented. 



Pretest-Post test Data 

The pretest, administered in Session 1, was composed of 75 questions 
designed to measure the level of achievement of the participants prior to 
the Presession so that gains in achievement could be measured. The posttest, 
administered in Session 19, also composed of 75 questions, was given to measure 
acquisition of the subject matter of the sessions. Table II presents the 
distribution of the pretest and post test scores. Table III indicates the 
measures of central tendency, and Table IV presents the adjusted frequency 
distribution which includes only the scores of participants who took both 
the pretest and posttest. 

An increase of participant demonstrated knowledge from pretest (mean=41) 
to posttest (mean=55) is demonstrated in graphic form, Table V, 

An analysis by objective is given in Table VI for the pretest and post- 
test, by indicating the percent of right responses. For every objective there 
was an increased percent of right responses. It should be mentioned that the 
large increase for objective 8 could be due to the fact that on the pretest 
many did not have enough time to finish. Table VII gives a more detailed 
analysis of each item of the pretest and posttest indicating the percent of 
participants who responded correctly. 



TABLE II 



DISTRIBUTION OF PRETEST AND POSTTEST SCORES 



P itest 



Post tost: 



Total Right 
75 



No. of Persons Total Right 

(N = 56) 

75 



No. of Persons 
(N = 52) 



64 

63 

62 

61 

60 

59 

58 

57 



56 1 

55 

54 

53 1 

52 1 

51 2 

50 

49 3 

48 2 

47 3 

46 2 

45 4 

44 1 

43 4 

42 4 

41 3 

40 3 

39 5 

38 2 

37 1 

36 1 

35 3 

34 4 

33 1 

32 

31 1 

30 2 

29 
28 

27 2 



64 

63 

62 

61 

60 

59 

58 

57 

56 

55 

54 

53 

52 

51 

50 

49 

48 

47 

46 

45 

44 

43 

42 

41 

40 

39 

38 

37 

36 

35 

34 

33 

32 

31 

30 

29 

28 

27 
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4 

4 

3 

4 

5 

3 
1 

4 



1 

1 



1 
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